Method, apparatus, and program product for developing and maintaining a comprehension state of a collection of information

ABSTRACT

Aspects of the disclosed technology enable a knowledge worker to easily and efficiently develop and maintain a comprehension state of a document collection. One aspect of the technology includes a methods, apparatus, and program products that alter a relationship data structure representing a comprehension state responsive to manipulation, in a workspace window, of a first instance-representation of a first separately-movable instance object representing a first entity/relationship object in the relationship data structure.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with Government support under MDA904-03-C-0404awarded by ARDA. The Government has certain rights in this invention.

CROSS-REFERENCE TO RELATED APPLICATIONS

The following United States Patent Applications have been filedherewith: U.S. patent application Ser. No. 11/426,915, inventor Bier,entitled ‘Method, Apparatus, and Program Product for Efficiently AddingEntities and Relationships to a Comprehension State of a Collection ofInformation’; U.S. patent application Ser. No. 11/426,919, havinginventors Bier and Ishak, entitled ‘Method, Apparatus, and ProgramProduct for Efficiently Defining Relationships in a Comprehension Stateof a Collection of Information’; and U.S. patent application Ser. No.11/426,925, inventors Bier and Ishak, entitled ‘Method, Apparatus, andProgram Product for Efficiently Detecting Relationships in aComprehension State of a Collection of Information’.

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC

(Not Applicable)

BACKGROUND

1. Technical Field

The disclosed technology relates to the field of sensemaking.

2. Background Art

Knowledge workers such as scientists, attorneys, intelligence analysts,private and public investigators/detectives and financial analysts allperform tasks that require reading and synthesizing information frommany documents. In such tasks, there is more information than a workercan hold in mind, so an essential element of the task is to record someof what has been learned in written or electronic form.

A knowledge worker often needs to track more information than can beheld in human memory. As a result, the knowledge worker generally usesan evidence file or notebook to record relevant information by storingentities and hand-typed notes about the information. The capturedinformation generally includes important relationships between theentities, between entities and other relationships, and betweenrelationships.

A computer can be used to add value to the notes. For example, theknowledge worker can use full text search to locate a note (if he/sheremembers words used in the note). In addition, if the notes includehypertext links, the worker can also use the links to re-find documentsthat have been previously read. However, the available computerassistance is limited because the computer does not have access toinformation about the relationships described in the documents, therelationships between those relationships, nor about which of therelationships are of greater or lesser interest to the knowledge worker.In addition, while a computer can search for text strings entered by theknowledge worker, it is unable to distinguish between text-snippets thatare of interest to the knowledge worker and those that are not.Furthermore, the detailed note-taking process is extremelytime-consuming and often the evidence filed does not include enoughinformation to allow computerized assistance.

The disclosed technology builds on work related to recording evidence,spatial hypertext, automatic highlighting, automating inference, readingrecommendations, and reading through multiple documents.

The disclosed technology differs from the Sandbox component of OculusnSpace (Wright et al., Advances in nSpace—the sandbox for analysis.Poster at the 2005 International Conference on Intelligence Analysis) inthat technology disclosed herein allows the knowledge worker to identifyand record specific entities and relationships from documents as well ashuman-readable entities, and also allows the knowledge worker toassociate a degree-of-interest value with each entity.

Single-mode snap-together operations have been used in the Niagarasystem (see: Good, L. E., Zoomable User Interfaces for the Authoring andDelivery of Slide Presentations. PhD dissertation, Department ofComputer Science, University of Maryland, Oct. 27, 2003). In Niagara,the knowledge worker can group text snippets by moving them closetogether. The technology disclosed herein extends this approach bysupporting two different kinds of grouping that result, respectively,from moving objects close together in vertical or horizontal directions,and by building a representation of all the entities and theirrelationships in the workspace.

Systems exist that employ automatic highlighting of text to aid readingand skimming. For example, the Scent Highlights component of the 3Booksystem automatically highlights words related to a query and sentencescontaining them to direct the reader's attention during skimming.Likewise, the Reader's Helper (see: Graham, J. The Reader's Helper: apersonalized document reading environment. Proceedings of the SIGCHIConference on Human Factors in Computing Systems (CHI '99), 1999, pages481-488) highlights phrases judged to be similar to a reader's topic ofinterest. The technology disclosed herein extends this approach byhighlighting both automatically-extracted entities and also phrases thathave been given a high degree of interest rating by the knowledgeworker.

Systems exist that automate the process of making inferences forintelligence analysis by using subgraph isomorphism to find suspiciouspatterns in a graph of entities and relationships (see: Coffman et al.,Graph-based technologies for intelligence analysis. Communications ofthe ACM, Volume 47, Number 3 (March 2004), 45-47). SRI's Link AnalysisWorkbench (see: Wolverton et al. LAW: A workbench for approximatepattern matching in relational data. In The Fifteenth InnovativeApplications of Artificial Intelligence Conference (IAAI-03), 2003)searches for entities in a graph that match a pattern of suspiciousbehavior either exactly or approximately. By contrast to these automatedapproaches, the disclosed technology provides interface tools for theknowledge workers to directly aid inference, based on whateverinformation the knowledge worker is viewing at any given moment.

Systems exist to assist a reader in selecting which document of adocument collection is to be analyzed next. For example, Woodruff et al.in Enhancing a Digital Book with a Reading Recommender (CHI 2000)described a Reading Recommender that analyses the relationships based ontextual similarity and co-citation between a set of documents and a listof documents read so far, and recommends new documents to examine. Bierin A document corpus browser for in-depth reading. Proceedings of theJoint Conference on Digital Libraries (JCDL), 2004, 87-96 discloses avisualization showing at a glance the most highly rated unreaddocuments, which act as an implicit recommendation. The disclosedtechnology builds on these approaches in at least two ways. First,because the knowledge worker assigns degree-of-interest values toindividual entities, recommendations are based on a relatively richmodel of the knowledge worker's interests. Second, the disclosedtechnology recommends both documents to read and also specificrelationships/entities to learn more about.

Systems exist for reading through a “trail” of documents (see: Bush, V.,“As We May Think.” The Atlantic Monthly, July 1945. Reprinted inInteractions, 3(2), 1996, pages 35-67). The technology disclosed hereinprovides a visualization of a set of trails, each of which correspondsto a query about an entity or set of entities.

The Oculus TRIST system (see: Jonker et al, Information triage withTRIST. 2005 International Conference on Intelligence Analysis), like thedisclosed technology, shows an icon per document and uses graphicalpresentation to distinguish read and un-read documents. Trails presentedby the technology disclosed herein differ from TRIST in that the trailsare automatically created responsive to the knowledge worker'smanipulations within the workspace window.

It would be advantageous to enable the knowledge worker to quicklyidentify particular phrases within a passage that correspond toimportant people, things, actions, or world events etc. and to provide adegree-of-interest value to these phrases. It would also be advantageousto suggest which electronic documents in a document collection toanalyze based on the knowledge worker's apparent interest as determinedfrom entities and their relationships and to assist the knowledge workerwhen making inferences.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a networked computer system in accordance with apreferred embodiment;

FIG. 2 illustrates an architecture that can be used with an embodiment;

FIG. 3 illustrates representation space architecture;

FIG. 4 illustrates an analysis process;

FIG. 5 illustrates an example of a workspace window;

FIG. 6 illustrates the correspondence between the relationshiprepresentation space 305 (as represented on the workspace window) and abelief graph;

FIG. 7 illustrates an electronic document preparation process;

FIG. 8 an electronic document presentation process;

FIG. 9 illustrates a text presentation window used to present a portionof an electronic document;

FIG. 10 illustrates a quick-click command process;

FIG. 11 illustrates a user-command dispatch process; and

FIG. 12 illustrates one relationship command process.

SUMMARY

Aspects of the technology disclosed herein enable a knowledge worker toeasily and efficiently develop and maintain a comprehension state of adocument collection. One aspect of the technology includes a methods,apparatus, and program products that alter a relationship data structurerepresenting a comprehension state responsive to manipulation, in aworkspace window, of a first instance-representation of a firstseparately-movable instance object representing a firstentity/relationship object in the relationship data structure.

Another aspect of the technology presents a workspace window responsiveto a relationship data structure that represents a comprehension stateincluding a presentation set of an ordered set of text strings from anelectronic document. The presentation set can include one or moreidentified strings. The workspace window can then receive a quick-clickcommand invocation on the one or more identified strings and modifiesthe relationship data structure by adding an entity/relationship objectto the relationship data structure responsive to the quick-click commandinvocation and the one or more identified strings.

Yet another aspect of the technology presents a workspace windowresponsive to a relationship data structure that represents acomprehension state including a presentation of a firstinstance-representation that represents a first separately-movableinstance object and a second instance-representation that represents asecond separately-movable instance object. This aspect enables the firstinstance-representation and detects when the firstinstance-representation is dropped within a threshold distance of thesecond instance-representation. When the first instance-representationis dropped, the technology identifies, responsive to the step ofdetecting, one of a plurality of spatial relationships between the firstinstance-representation and the second instance-representation.Responsive to the identified spatial relationship, the technologyselects an operation, and responsive to the operation modifies acomposite object in the relationship data structure. The compositeobject incorporates an entity/relationship object that is represented bythe first separately-movable instance object and an entity/relationshipobject represented by the second separately-movable instance object.After the composite object is modified, an instance-representation thatrepresents the composite object is presented in the workspace window.

Still another aspect of the technology presents a workspace windowresponsive to a belief graph and a relationship data structure thatrepresents a comprehension state by presenting aninstance-representation of a first separately-movable instance objectthat represents a first entity/relationship object in the relationshipdata structure wherein the instance-representation of the firstseparately-movable instance object can be selected.

One aspect of the disclosed technology is a computerized tool thatassists a knowledge worker in selecting entities (that can becategorized and rated) from undifferentiated text and maintainingrelationships and inferences about the undifferentiated text. Once theentity information is available, the disclosed technology can assist theknowledge worker with finding and re-finding information about relevantrelationships, provide recommendations of which relationships, passages,and electronic documents are likely to contain information of interest,provide aids to discovering relationships between the entities, and canprovide reading aids that draw the knowledge worker's attention toimportant words, phrases, and passages in the document collection.

The embodiments disclosed herein use an object oriented programmingparadigm. In such a paradigm, an object is an association betweenprogrammed methods and the data structures defined by a class and theinstantiated storage that represents an object of the class. Classes canhave superclasses and subclasses. One skilled in the art will understandthat although the disclosure is cast within an object oriented paradigm,the techniques disclosed are applicable to other programming paradigms.

One embodiment of the disclosed technology provides improvedcomputerized tools to analyze, organize, and visualize largeheterogeneous information resources (such as a large documentcollection) and assist in discovering and understanding the informationand information relationships within the resources that are of interestto a knowledge worker. By using such tools, knowledge workers are betterable to mine, find, remember, and understand the information of interestburied within the information resources, and thus to make more informeddecisions based on the available information. Some of these capabilitiesinclude information visualizations that aid inference of relationshipsbased on relationships that are indirectly represented, aids to findingand re-finding relevant entities (such as person names, organizationnames, telephone numbers, addresses, city names, country names, statenames, pathogen identifications, explosive types, currency values, etc.)and the relationships between the entities, capabilities that recommendwhich entities, passages, and documents the knowledge worker is likelyto find useful to evaluate, aids for identifying relationships betweenentities, and reading aids that draw the knowledge worker's attention toimportant words, phrases, and passages related to the relationshipsand/or entities. These computerized tools are especially useful tointelligence analysts, lawyers, technology analysts, policeinvestigators, private investigators, and those in the medical,financial, and research industries.

An entity is an information snippet that can carry categorized meaning.Thus an entity can be classified as a person, place, or thing, anaction, a time, statute, citation, condition, medical condition, orother classification desired by the knowledge worker.

One embodiment enables knowledge workers to quickly capture entityand/or relationships from an electronic document using combinations ofpoint-and-click and automatic entity extraction operations. Eachcaptured entity is stored in a representation space. The knowledgeworker quickly establishes relationships between one entity and anotherentity (or previously established relationship) by directly manipulatingrepresentations of the entities and relationships on a Graphical UserInterface (GUI). In addition, the knowledge worker can create differentstrength relationships using GUI manipulations as well as by specifyingthe knowledge worker's degree-of-interest in the relationship and/or theentity.

In one embodiment, the GUI combines traditional GUI operations andentity extraction technologies to construct an explicit model ofrelationships between entities (such as people, places, organizations,phone numbers, etc.), relationships and combinations thereof. By usingtools enabled by the model, the knowledge worker can better mine,discover, and be reminded of relationships and/or entities; can betteridentify and locate electronic documents in the document collection thatsupport relationships and/or entities; can better discover relationshipsand/or entities; and can better identify electronic documents,relationships, and/or entities for follow-on evaluation. Thus, aspectsof the technology assist the knowledge worker in finding and returningto interesting documents. One embodiment can be implemented using ageneral purpose computer (for example, one such as shown in FIG. 1).

DETAILED DESCRIPTION

FIG. 1 illustrates a computer system 100 that can incorporate anembodiment. The computer system 100 includes a computer 101 thatincorporates a CPU 103, a memory 105, and a network interface 107. Thenetwork interface 107 provides the computer 101 with access to a network109. The computer 101 also includes an I/O interface 111 that can beconnected to a user interface device(s) 113, a storage system 115, and aremovable data device 117. The removable data device 117 can read acomputer-usable data carrier 119 (such as a fixed or replaceable ROMwithin the removable data device 117 itself (not shown); as well as acomputer-usable data carrier that can be inserted into the removabledata device 117 itself (such as a memory stick, CD, floppy, DVD or anyother tangible media) that typically contains a program product 121. Theuser interface device(s) 113 can include a display device 125 and userinput devices (not shown). The storage system 115 (along with theremovable data device 117), the computer-usable data carrier 119, and(in some cases the network 109) comprise a file storage mechanism. Theprogram product 121 on the computer-usable data carrier 119 is generallyread into the memory 105 as a program 123 which instructs the CPU 103 toperform specified operations. One skilled in the art will understandthat not all of the displayed features of the computer 101 need to bepresent for the all embodiments that implement the techniques disclosedherein.

FIG. 2 illustrates an architecture 200 that can include a documentdatabase and structure storage 201 that stores a document collection aswell as state information about the relationships and/or entities foundin the document collection. A document installation process 203 installselectronic documents into the document collection and can performsubsequently described processing to discover entities that could be ofinterest to a knowledge worker. A relationship discovery process 205 canbe used by a knowledge worker 207 to discover and record therelationships and/or entities from the document collection that are ofinterest to the knowledge worker. The relationship discovery process 205can also include tools to help the knowledge worker remember previouslydiscovered relationships and/or entities and thus improve the knowledgeworker's efficiency.

FIG. 3 illustrates a representation space architecture 300 that can beused in an embodiment. A knowledge worker can interact with a graphicaluser interface 301 to manipulate entity/relationship objects (thatrepresent relationships and entities) to discover and record newrelationships between the entity/relationship objects, and/or previouslydiscovered relationships. The user does this by manipulating presentedinstance-representations of separately-movable instance objects residingin an instance representation space 303 (as is subsequently describedwith respect to FIG. 5). The separately-movable instance objectsrepresent entity/relationship objects in a relationship representationspace 305. The relationship representation space 305 maintains one ormore relationship data structures about the entities and theirrelationships from the document collection and/or from the knowledgeworker's knowledge, belief, or hypothetical facts or hypotheses (as isalso subsequently described with respect to FIG. 5). As the knowledgeworker manipulates instance-representations the relationship datastructures are altered accordingly. A belief representation space 307includes a belief graph (as is subsequently described with respect toFIG. 6) that models the relationship data structures in the relationshiprepresentation space 305 and, among other purposes, enables theknowledge worker to use predictive/inference tools responsive to thestructure of the relationship representation space 305 to help directfuture investigation of the document collection. The disclosedtechnology also provides tools to achieve ranking, scoring, or analysisof the document collection. The representation space architecture 300(in particular, the relationship representation space 305 and the beliefrepresentation space 307) maintains a comprehension state of theknowledge worker's understanding of, and inferences from, the documentcollection. The comprehension state is developed by the knowledgeworker's manipulating instance-representations representingentity/relationship objects in the relationship representation space305.

In the instance representation space 303 each entity/relationship objectin the relationship representation space 305 can be represented by oneor more separately-movable instance objects in the instancerepresentation space 303. Thus, a entity/relationship object in therelationship representation space 305 can be represented multiple timeson the GUI via multiple separately-movable instance objects, each ofwhich represent the entity/relationship object in the relationshiprepresentation space 305. One skilled in the art would understand how toimplement equivalent embodiments using other object-oriented programmingor procedural programming methodologies. In the following, one skilledin the art will understand that the term “selected entity” means, forthe embodiment described herein, that the knowledge worker has selectedan instance-representation of a separately-movable instance object fromthe instance representation space 303 that represents anentity/relationship object in the relationship representation space 305.The entity/relationship object can represent either an entity or arelationship and the term selected entity implies either.

FIG. 4 illustrates an analysis process 400 that can be executed by acomputer to assist the knowledge worker with the problem of capturingand classifying entities and their relationships from a documentcollection. The analysis process 400 initiates at a ‘start’ terminal 401responsive to invocation by, for example, the knowledge worker. Onceinitiated, the analysis process 400 continues to an ‘access documentcollection’ procedure 403 that opens access to the electronic documentsin the document collection. In addition, a ‘restore representationspaces’ procedure 405 initializes or retrieves a previously stored stateof the representation spaces from storage. A ‘present workspace window’procedure 407 can then present instance-representations on a workspacewindow 500 of separately-movable instance objects from the instancerepresentation space 303 that, in turn, represent entity/relationshipobjects in the relationship representation space 305 (as is subsequentlydescribed with respect to the GUI of FIG. 5). A ‘manipulate presentedinstances’ procedure 409 allows the knowledge worker to manipulate theinstance-representations in the workspace window 500. As the knowledgeworker manipulates the instance-representations to select entities anddefine their relationships, the analysis process 400 continues to a‘modify representation spaces’ procedure 411 that modifies therepresentation spaces responsive to the knowledge worker'smanipulations. Then the analysis process 400 loops back to the ‘presentworkspace window’ procedure 407 to present updatedinstance-representations in the workspace window 500 responsive to themanipulations. The analysis process 400 continues until it is terminatedby the knowledge worker or in response to a termination event.

The GUI can present information from the relationship representationspace 305 and/or the belief representation space 307 to the knowledgeworker. The separately-movable instance objects representing objects inthe relationship representation space 305 can be presented ascorresponding instance-representations within the workspace window 500such as illustrated by FIG. 5. The workspace window 500 can include anevidence panel 501 as well as other panels such as a system suggestionspanel 503, a trails panel 505, and an entity/relationship objectinspector panel 507.

The knowledge worker can use the workspace window 500 as a memory aid byusing the evidence panel 501, the system suggestions panel 503, and thetrails panel 505 to help remember what was previously learned from thedocument collection. In particular, each presentedinstance-representation (which represents an entity/relationship object)serves as a reminder for the knowledge worker of the relationships of,and the importance of, the relationships and/or entities contained inthe entity/relationship object.

The evidence panel 501 presents instance-representations ofseparately-movable instance objects in the instance representation space303 that represent entity/relationship objects in the relationshiprepresentation space 305. The knowledge worker can use the evidencepanel 501 to define/modify the relationships between theentity/relationship objects by manipulating the instance-representationspresented on the GUI.

The system suggestions panel 503 can present a list of recommendedentity/relationship objects that may be of interest to the knowledgeworker. This list results from analysis of the belief representationspace 307 as is subsequently described with respect to FIG. 6. Thetrails panel 505 can identify electronic documents in the documentcollection which include text matching the entity(s) in theentity/relationship object represented by the selectedinstance-representation. The entity/relationship object inspector panel507 can present, in an easy-to-scan single column, a list of otherentity/relationship objects relevant to the entity/relationship objectsrepresented by the selected instance-representation.

An entity/relationship object in the relationship representation space305 can be multiply represented in the workspace window 500. Eachrepresentation of any given entity/relationship object, corresponds to aseparately-movable instance object in the instance representation space303 that presents an instance-representation in the workspace window 500of the entity/relationship object. Thus, multiple representations of anentity/relationship object can be presented on the workspace window 500to establish relationships. As is subsequently described with respect toFIG. 11, a command can be provided to the knowledge worker to make a newcopy of any separately-movable instance object. That copy then acts likeany other instance-representation representing that specificentity/relationship object, and the new instance-representation can bepositioned on the workspace window 500.

The knowledge worker can place an instance-representation thatrepresents one entity/relationship object within one or more otherinstance-representations to specify relationships between theentity/relationship objects so bundled. For example, the knowledgeworker can manipulate the relationship representation space 305 bydropping an instance-representation representing an entity object ontoan instance-representation that represents an evidence bundle object tobundle the entity object with other entity/relationship objects alreadycontained in the receiving evidence bundle object. Such manipulation canbe performed to add an entity/relationship object to separate evidencebundle objects, within instance-representations of separate beliefstatement objects, and both in and out of evidence bundle objects, andin and out of belief statement objects. Because allinstance-representations of separately-movable instance objectsrepresenting an entity/relationship object represent the sameentity/relationship object, the knowledge worker can use multipleinstance-representations to relate the entity/relationship object to anynumber of other entity/relationship objects.

The knowledge worker can select an instance-representation using anycommonly known GUI interaction device and/or method. In one embodiment,when an instance-representation is selected, the selectedinstance-representation and all instance-representations representingrelated and/or referenced entity/relationship objects in therelationship representation space 305 highlight in the evidence panel501. In addition, all instance-representations representing compositeobjects that contain the corresponding entity/relationship object(and/or entity/relationship objects having a linked relationship withthe selected entity/relationship object) can also highlight. Thus, theknowledge worker can be quickly reminded of and/or can quickly locateentities and relationships relevant to the entity/relationship objectrepresented by the selected instance-representation. If multipleentity/relationship objects are selected, each instance-representationrepresenting only one of the selected instance-representation can behighlighted with a color representing the color of the representedselected instance-representation. However, a different availablehighlight can be used when an instance-representation represents morethan one of the selected instance-representations.

Instance-representations can also indicate a degree-of-interest value byicon, color, or other designation. For example, the degree-of-interestvalue displayed may be the value set by the knowledge worker.Alternatively, the degree-of-interest value can be determined from theknowledge worker's specified degree-of-interest values as adjusted bythe belief graph (such as by spreading activation).

Highlighting can be performed by changing color, shape, font, size,style, brightness, or any other way of distinguishing oneinstance-representation from another. Thus, one skilled in the art willunderstand that the term “highlighting” includes the capability ofdisplaying the highlighted text with a distinctive graphical property.For example, in one embodiment, instance-representations that areselected can be highlighted by applying a more distinctive highlightwhile instance-representations that are not selected can be dimmed,grayed out, or otherwise made less distinctive.

An entity/relationship object can be an entity object, a comment object,an electronic document object, a document page object, a compositeobject, an evidence bundle object, a belief statement object, etc.Relationships can be defined by some of these objects. For example, theevidence bundle object represents an evidence relationship between thebundled objects where the relationship strength is relatively weak; thebelief statement object represents a statement relationship where thebundled objects have a stronger relationship than that of the evidencerelationship; entities can be chained together to form a couplingrelationship between the chained entities such that both entities arealways presented whenever the first entity of the couple is presented.In some embodiments coupling is not symmetrical, while in others it issymmetrical. One example of coupled entities is the circumstance ofcoupling a person's contact information (such as a telephone number, ane-mail address, a mailing address, a work address, home address, etc.)to the person's name, such that the coupled name is presented wheneverthe contact information is presented.

Entity objects can be added to the relationship representation space 305by dragging their respective instance-representation from theentity/relationship object inspector panel 507 or the system suggestionspanel 503 to the evidence panel 501. In some embodiments, the knowledgeworker can select a document page object instance 508 to invoke adocument reader tool (see the subsequent description related to FIG. 9)to display the corresponding document page for the knowledge worker'sreference. Entity objects can be added to the relationshiprepresentation space 305 by copying text (to be captured as an entity)from the document reader tool (as is subsequently described with respectto FIG. 7) using traditional cut-and-paste and drag-and-drop GUIoperations as well as inventive “quick-click” operations that aresubsequently described with respect to FIG. 10. In addition, theknowledge worker can type or copy any text string into the evidencepanel 501 to create an entity.

In addition to manipulating instance-representations that represententity objects, the knowledge worker can type or copy any text stringinto the evidence panel 501 to create a comment object in therelationship representation space 305. Comment objects help label and/ororganize the entities and relationships in the relationshiprepresentation space 305. Presented instance-representations thatrepresent comment objects are similar to instance-representations ofentity/relationship objects, but, in one embodiment, do not present aentity-categorization icon and can be displayed with different coloredtext and background (such as a comment object instance 509). Commentobjects in the relationship representation space 305 have similarcharacteristics to entity objects and can be included in compositeobjects but in one embodiment are not used to establish relationshipsbetween entities.

The system suggestions panel 503 presents a list of entity/relationshipobjects ordered responsive to the belief representation space 307 (as issubsequently described with respect to FIG. 6). Thus, the systemsuggestions panel 503 can provide recommendations to the knowledgeworker of which entity objects are likely to be worth subsequentinvestigation.

In some embodiments the list in the system suggestions panel 503 can besorted first by the entity-categorization (for example, such that allentity objects of type “person” precede all entity objects of type“phone-number” etc.), and then by the spreading activation scores of thefact nodes in the belief representation space 307 representing theentity objects in the relationship representation space 305. In someembodiments the system suggestions panel 503 can include aninstance-representation of one or more document page objects at the topof the panel that can, but need not be, associated with the currentlyopen pages in the document reader tool. In some embodiments, documentpage objects may also appear in a document page object section (notspecifically labeled in FIG. 5) of the system suggestions panel 503, andordered responsive to the belief representation space 307.

The trails panel 505 presents information about selectedinstance-representations of one or more entity/relationship objects byidentifying electronic documents in the document collection that containthe selected entities. One embodiment presents one icon for each suchelectronic document and uses icon color and other graphical propertiesto indicate facts about the electronic document associated with the icon(for example, but without limitation, such as whether or not theknowledge worker has read the electronic document, whether or not theelectronic document is open for reading, the number of pages in theelectronic document, how highly the knowledge worker has rated theelectronic document compared to other electronic documents in thedocument collection, and how many times the selected entities appear inthe electronic document). In addition, the trails panel 505 state can beresponsive to queries by the knowledge worker targeting, for example, asingle entity, such as “Joe Jones”, or multiple entities such as “JoeJones” AND explosives AND “Harry Hill”.

The entity/relationship object inspector panel 507 provides theknowledge worker with a summary of the information contained in orrelated to the entity/relationship object of a selectedinstance-representation. This information can include one or more of thetext strings that represent the selected entity, any text strings thatserve as aliases for the selected entity, references to any electronicdocuments that contain the selected entity, any belief statement objectreferencing the selected entity, other relationships fromentity/relationship objects that are contained in the same evidencebundle object as the selected entity, and other entities fromentity/relationship objects that are sourced from the same electronicdocuments as the selected entity.

The knowledge worker can select or operate on an entity/relationshipobject using any traditional GUI selection tool/method (for example, byclicking on the presented instance-representation that represents theentity/relationship object) or using quick-click commands assubsequently described with respect to FIG. 10, FIG. 11, and FIG. 12.

As previously described with respect to FIG. 3, the relationshiprepresentation space 305 can be presented in the evidence panel 501through the presentation of instance-representations ofseparately-movable instance objects representing entity/relationshipobjects within the relationship representation space 305. Such apresentation can include a belief statement object (represented by abelief statement object instance 510) that defines a strong relationshipbetween the members of the belief statement object, a evidence bundleobject (shown by an evidence bundle object instance 511) that defines aless strong relationship between its members, a document page objectthat can be represented as a collection of at least one document pageobject such as the document page object instance 508; and a commentobject that can be represented by the comment object instance 509.

Entities can also be linked together to form relationships (symmetricalor asymmetrical). Once such linked relationship is a couplingrelationship. A coupling relationship instance 517 (in one embodiment)couples two entity/relationship objects such that when aninstance-representation of the primary member of the couple ispresented, an instance-representation of the secondary member of thecouple is presented adjacent to that of the primary. Another linkedrelationship is an alias relationship (not shown) that defines that theentities in the alias relationship are the same. Entities linked by analias relationship have the strongest relationship and are representedby a single fact node in the belief representation space 307.

Selecting one instance-representation can cause otherinstance-representations that reference the selectedinstance-representation to highlight (as indicated by hashing in FIG. 5responsive to a selection of a ‘Henry Hill’ instance-representation519).

Entity/relationship objects that are related to each other in some waycan be placed within an evidence bundle object to create a relationship.For example, an evidence bundle object instance 521 in FIG. 5 indicatesthat “Joe Jones” is, in some way, connected to a phone number, acompany, an FBI report document page, a date, a bank, etc.

In one embodiment, the class of an entity/relationship object is asuperclass of the classes used to instantiate an entity object (thatrepresents the entity selected by the knowledge worker and can includean entity-categorization and/or a degree-of-interest value), a commentobject (that allows the knowledge worker to provide additionalinformation that does not affect the relationships between theentities), a document page object (that maintains information about apage of an electronic document object), an electronic document object(that represents an electronic document from the document collection),and a composite object (that defines relationships between theentity/relationship objects bundled within the object). The class of thecomposite object is a superclass of the classes used to instantiate anevidence bundle object and a belief statement object (that representdifferent strength relationships between the entity/relationship objectsbundled within the composite object).

FIG. 6 illustrates a representation space relationship 600. As has beenpreviously discussed, the belief representation space 307 includes abelief graph 601 (generally not displayed to the knowledge worker, butwhich is provided in FIG. 6 for explanatory purposes). The knowledgeworker identifies entities and relationships by manipulatinginstance-representations on the evidence panel 501. The presentations ofthese instance-representations can be generated from separately-movableinstance objects in the instance representation space 303. Theentity/relationship objects in the relationship representation space 305can be changed responsive to the knowledge worker's manipulation. Theinformation from the relationship representation space 305 then can beused to generate/modify the belief graph in the belief representationspace 307.

A presentation of the belief graph 601 is generally not useful to theknowledge worker because such a representation of the beliefrepresentation space 307 quickly leads to considerable screen clutterwhich makes it very difficult for the knowledge worker to analyze,manipulate, edit or use the belief representation space 307.

The relationship representation space 305 represents entities andrelationships that the knowledge worker has determined to be important.Information about an entity can be stored in an entity object in therelationship representation space 305. Relationships between theentities, as determined or verified by the knowledge worker, arecaptured as composite objects in the relationship representation space305. The evidence panel 501 in FIG. 6 can be used to presentinstance-representations of separately-movable instance objects from theinstance representation space 303 which represent theentity/relationship objects in the relationship representation space305. The relationship representation space 305 can be used to define thebelief graph 601. The belief graph 601 represents the strength of therelationships between the entity objects in the relationshiprepresentation space 305.

The belief graph 601 can be an undirected graph having fact nodesrepresenting entity objects (such as a Timbuktu entity fact node 603)and document page objects (not shown). Edges between fact nodes can beweighted by the strength of the relationship between pairs of fact nodesas determined from the interrelationships of the entity objects in therelationship representation space 305. Comment objects in therelationship representation space 305 do not become fact nodes, butbecome an edge property in the belief graph.

FIG. 6 also illustrates instance-representations of separately-movableinstance objects representing a first evidence bundle object 605 and asecond evidence bundle object 607 in the evidence panel 501 togetherwith a visualization of the belief graph 601, which represents therelationships between the entities of the evidence bundle objects.

When generating the belief graph 601 from the relationshiprepresentation space 305, a weighted edge can be placed in the beliefgraph 601 between any two fact nodes that have a known relationship. Theweighted edge represents a relationship between entity objectsrepresented by the fact nodes connected by the weighted edge. Inparticular, a weighted edge can be placed between all pairs of factnodes that share a composite object (such as an evidence bundle objector a belief statement object) in the relationship representation space305. In addition, weighted edges may be added into the belief graph 601responsive to information external to that provided via the evidencepanel 501. For example, a weighted edge can be placed between fact nodesrepresenting two entity objects if the text strings of the entities thatwere used to define the entity objects (or of their coupled objects) arenear each other in one or more electronic documents in the documentcollection. No weighted edges attach to a fact node that corresponds toentity object that is not contained within a composite object orotherwise linked to other objects such as by a coupling relationship, analias relationship, or co-occurrence in documents (as in the case of anexplosives object 609). Entities within an alias relationship arerepresented by a single fact node in the belief graph 601.

Weighted edges can be given higher weight values if the joined factnodes have a stronger relationship (for example, as determined from thetype of composite object that contains the entity objects). For example,weighted edges that join fact nodes that represent entity objects thatshare a belief statement object can be weighted more heavily thanweighted edges that join fact nodes that represent entity objects thatshare an evidence bundle object. Weighted edges joining fact nodes thatrepresent entity objects that share multiple evidence bundle objects ormultiple belief statement objects can be weighted more heavily thanweighted edges joining fact nodes that represent entity objects thatshare only a single composite object. Weighted edges joining fact nodesthat represent entity objects in a coupling relationship have evenstronger weights. Entity objects in an alias relationship can berepresented by a single fact node.

The belief graph 601 can also represent a relationship betweenco-occurrences of text represented by entity objects in separateelectronic documents in the document collection. The knowledge workercan explicitly create a relationship between the entity/relationshipobjects from the separate electronic documents. In this situation theweighted edge can be more heavily weighted than weighted edges joiningentity/relationship objects from the same electronic document. If theentity from one electronic document is the same as an entity fromanother electronic document, the knowledge worker can put the twoentities into an alias relationship.

Once the belief graph is constructed in the belief representation space307, the knowledge worker can then use tools enabled by the belief graphto provide recommendations and inference aids, such as was previouslydescribed with respect to the system suggestions panel 503 of FIG. 5.

In one embodiment, the ordering of the entity object in the systemsuggestions panel 503 can be determined using a spreading activationalgorithm over the belief graph where the initial activation of eachfact node in the belief graph can be computed from a degree-of-interestvalue property of the represented entity object (either as set bydefault or as explicitly specified by the knowledge worker). Afterapplication of the spreading activation algorithm (or other inferenceengine algorithm), the highest scoring fact nodes will include thosethat represent entity objects that were explicitly rated highly by theknowledge worker, those that were linked from highly ratedentity/relationship objects by the shortest path of weighted edges,those with the most highly weighted edges to fact nodes that representhighly rated entity/relationship objects, those with multiple weightededges to fact nodes that represent highly rated entity/relationshipobjects, or a combination of these factors.

Once the relationship representation space 305 is constructed from theentity/relationship objects, the knowledge worker can then use tools inthe evidence panel 501 to quickly find, remember, and developrelationships between entities (including tools to assist the knowledgeworker when scanning text for relationships such as provided by thedocument reader tool (that is discussed with respect to FIG. 8 and FIG.9).

One way the technology assists the knowledge worker is thatinstance-representations of separately-movable instance objects caninclude a graphical indicator of the entity-categorization of theinformation or relationship contained in, or represented by, therepresented entity/relationship object. Examples ofentity-categorizations for the entity object include a person name, anorganization name, a telephone number, an address, a city, a country, astate, a pathogen, a type of explosive, a currency value, etc.; or otherinformation type. A corresponding graphical indicator for eachentity-categorizations can be presented as well as a default graphicalindicator for entity/relationship object that do not have a specifiedentity-categorization or that have an unknown entity-categorization.

One embodiment uses icons with the instance-representation to identifythe entity-categorization of the entity. The entity-categorization canbe specified by the knowledge worker when inserting the entity into therelationship representation space 305 or can be determined (orsuggested) by the document installation process 203 (or other processfor applying the rule database to the electronic document).

The knowledge worker, by manipulating the instance-representationspresented by the GUI, can manipulate the separately-movable instanceobjects to establish relationships between entity objects (representedby the separately-movable instance object) and other relationshipsdefined in the relationship representation space 305. For example, byplacing a mouse cursor, pen tip, stylus tip switch, or other pointingdevice (or any other GUI selection tool or method) over one of theinstance-representations, pressing a button, moving the pointing device,and releasing the button, the knowledge worker can position eachinstance-representation of a separately-movable instance object todefine relationships in the relationship representation space 305between the entity/relationship objects represented by theseparately-movable instance object. These relationships can be used toweight edges between fact nodes in the belief graph.

Thus, relationships are created by the knowledge worker manipulatingseparately-movable instance objects to define an entity/relationshipobject establishing the relationship between multiple entity objects,between an entity object and an entity/relationship object, as well asbetween two entity/relationship objects (such as between compositeobjects and combinations of entity/relationship objects). Therelationships can be manually or semi-automatically created by theknowledge worker, or automatically by the knowledge worker invoking acommand such as a quick-click command (as is described with respect toFIG. 10). In addition, FIG. 12 illustrates a snap-together commandmechanism that allows the knowledge worker to even more quickly createrelationships between the entities.

By manipulating the instance-representations on the GUI, the knowledgeworker creates a comprehension state of important entities and theirrelationships. The comprehension state reflects the currentunderstanding of the document collection by the knowledge worker and/orhis/her co-workers.

While the knowledge worker selects the entities from electronicdocuments for incorporation into the evidence panel 501; the disclosedtechnology also provides tools to assist with this task. These tools caninclude an initial-categorization tool that uses rules to identifypotential entities and specify a default categorization.

FIG. 7 illustrates an electronic document preparation process 700 thatcan be invoked by the document installation process 203 of FIG. 4; thatinitiates at a ‘start’ terminal 701 and continues to an ‘open electronicdocument’ procedure 703 to add/access an electronic document to/from thedocument collection. The electronic document includes an ordered set oftext strings. The electronic document preparation process 700 can alsoopen a rule database with an ‘open rule database’ procedure 705. An‘identify identified string’ procedure 707 can apply rules from the ruledatabase to the electronic document to recognize identified stringswithin the electronic document and a ‘save identified stringinformation’ procedure 709 saves sufficient information to quicklylocate the identified strings. The electronic document preparationprocess 700 exits through an ‘end’ terminal 711.

In some embodiments, the electronic document preparation process 700 canbe used to process an electronic document to recognize the identifiedstrings as the electronic document is being displayed. Furthermore, therules can automatically assign an entity-categorization and/ordegree-of-interest value to one or more of the identified strings. Theknowledge worker can modify the entity-categorization and/ordegree-of-interest value as desired.

Some embodiments enable the knowledge worker to specify an importanttext string to be added to the rules and provides the option ofre-processing the document collection to incorporate the new rule.

FIG. 8 illustrates an electronic document presentation process 800 thatcan be invoked by the knowledge worker, initiates at a start terminal801 and continues to a ‘present electronic document portion’ procedure803 that presents some portion (the presentation set) of the electronicdocument selected by the knowledge worker. The electronic documentpresentation process 800 can be implemented within a document readertool. An ‘iterate identified strings’ iterative procedure 805subsequently or simultaneously locates each identified string in thepresentation set and for each iterated identified string, candistinguish that string from non-identified strings via a ‘distinguishidentified string’ procedure 807 (the string can be distinguished, forexample, by highlighting, change of font style, change of text color,etc.). After the presented identified strings have been distinguished,an ‘add user-selected distinguished identified string’ procedure 809enables the knowledge worker to designate a string (identified or not)as an entity and to directly or indirectly copy the selected string(s)to the evidence panel 501, which defines an entity object in therelationship representation space 305 that contains the selected string.Copying the string can be done, for example, by traditionaldrag-and-drop or cut/copy-and-paste operations as well as by the use ofquick-click commands as is subsequently described with respect to FIG.10.

The electronic document presentation process 800 can terminate (notshown) after the knowledge worker has finished defining entities or canloop to the ‘present electronic document portion’ procedure 803 topresent additional portions of the electronic document. Additionalportions of the electronic document can be presented by the knowledgeworker changing the presentation set by scrolling or paging through theelectronic document. Mechanisms for selecting identified strings aresubsequently described with respect to FIG. 10. Select-drag-and-droptechniques for specifying a non-identified string as a selected stringare well known in the art. In some embodiments, processing directedtowards identified strings can be disabled if the knowledge workerexplicitly selects the selected string.

The presented portion of the electronic document makes up a presentationset of the ordered set of text strings where one of the one or moreidentified strings within the presentation set is distinguished from asecond subset of the presentation set (for example those strings thatare not identified by the rules in the rule database). Each of theidentified strings can also be (manually or automatically) assigned acategorization to identify what the identified string represents (suchas a person, building, telephone number, address, etc.). Thecategorization can be used as the entity-categorization in an entityobject.

For example, as shown by FIG. 9, a text presentation window 900(presenting a portion of a fictitious FBI report) contains an identifiedtext string 901 that uses yellow highlighting to distinguish theidentified text string 901 from a non-identified text string 903. InFIG. 9 yellow highlighting is represented by a dashed box placed aroundthe words that would be highlighted on the GUI. Red highlighting can beused to indicate entities of high interest to the knowledge worker. Redhighlighting in FIG. 9 is indicated by boxes with bold borders. Todetermine which entities are of high interest, the document reader toolcan access the belief graph when preparing the presentation of a portionof a document page object to detect entities that have a highdegree-of-interest value. This enables the knowledge worker to morequickly scan an electronic document for information relevant to theknowledge worker or information that is included in the comprehensionstate.

One skilled in the art will understand that some embodiments apply therules to the presented portion of the electronic document as thatportion is being displayed while other embodiments can preprocess thecomplete electronic document with the rules prior to any portion of theelectronic document being displayed. Still other embodiments canpreprocess the electronic document by applying the rules prior toexecution of the electronic document presentation process 800.

The rule database can include rules that identify strings that areimportant to the knowledge domain of the knowledge worker. For example,if the knowledge worker is an intelligence analyst, the rule databasegenerally would include rules to identify the name of a person, the nameof an organization, an address, a telephone number, a city, a country, astate, a pathogen, a type of explosive, and so on. If the knowledgeworker were a patent attorney, the rules could identify terms or phrasesused in the patent statute, the PTO rules, invention components etc. Therules can also assign a categorization to the identified string. Thus,the identified string “John Smith” can be associated with thecategorization “person”; the identified string “703-555-1212” can beassociated with the categorization “phone-number”; and so on. When theidentified string is copied to the evidence panel 501 to define it as anentity in the relationship representation space 305, the categorizationcan be included in the separately-movable instance object as anentity-categorization.

Traditional methods of identifying and copying text to the evidencepanel 501 are cumbersome and time consuming. Nevertheless, as previouslydiscussed, the knowledge worker can insert an entity into the evidencepanel 501 by selecting any text string and placing it into the evidencepanel 501 using cut-and paste or drag-and-drop operations Variations ofthe paste operation allow the selected string to be added either as anentity or a comment; added as an entity object in the relationshiprepresentation space 305 and can be included within a composite objector linkage to create a relationship. In addition, aninstance-representation of a document page object can be displayed nearthe workspace window 500 and the knowledge worker can drag one or morecopies of the document page object into the evidence panel 501 where itcan then be positioned in the evidence panel 501 or added to beliefstatement objects and/or evidence bundle objects. Furthermore, theknowledge worker can click at any position in the evidence panel 501 andinsert a new entity object or comment object at that position. Theknowledge worker can then input the text of the new entity object orcomment object. Manual drag-and-drop or cut-and-paste operations areslow and tend to distract the knowledge worker from the analysis of thedocument collection.

Some embodiments include a quick-click command to speed the process ofinserting entities into the relationship representation space 305. Toinvoke one of the available quick-click sub-commands, the knowledgeworker can hold down a button (such as the Shift, Option, ALT, CTRL,etc. or a combination of buttons on a standard computer keyboard, or usea gesture, or other well-known GUI command invocation technique) whileclicking on the word or phrase that is to be added as an entity. As soonas the click is complete, a copy of the selected word or phrase can beadded to the relationship representation space 305 and can be presentedin the evidence panel 501 through the instance representation space 303.Other quick-click sub-commands can be specified by using different, ordifferent combinations of, buttons or other user controls.

FIG. 10 illustrates a quick-click command process 1000 that theknowledge worker can invoke to quickly and efficiently select and entera string from, for example, the text presentation window 900 into theevidence panel 501 (and hence create an entity object orentity/relationship object). The quick-click command process 1000 can beinvoked responsive to the knowledge worker performing an action thatposts a “quick-click” command event (or by any other method forperforming a quick-click command invocation).

Once the commend event is posted, the quick-click command process 1000initiates at a ‘start’ terminal 1001 and continues to a ‘receivequick-click command’ procedure 1003 that receives information about theposted command event. Once the command information is received, a‘determine quick-click sub-command’ procedure 1005 uses that informationto determine the specified quick-click sub-command. The command can bethen dispatched by a ‘select on sub-command’ procedure 1007 that selectsthe procedure responsible for effectuating the quick-click sub-command.If the knowledge worker has selected specific text in the textpresentation window 900—that is, if the knowledge worker has designateda user-selected subset of the ordered set of text strings from theelectronic document (whether or not the user-selected subset is orcontains an identified string or non-identified string) and theknowledge worker's cursor position intersects the user-selected subsetthen the quick-click command process 1000 continues to a ‘user-selectedrange as entity’ procedure 1009 that sets the selected string to be acopy of the user-selected subset.

Next, an ‘add entity(s)’ procedure 1011 creates an entity object (thatcan contain entity-categorization) in the relationship representationspace 305 (and modifies the structure of the belief representation space307 and the instance representation space 303 corresponding to theaddition of the entity object in the relationship representation space305). In one embodiment, if the selected string is already representedby a pre-existing entity object from the same electronic document, aseparately-movable instance object representing the pre-existing entityobject can be inserted into the instance representation space 303. Ifthe selected string is sourced from a different electronic document thanthat of a pre-existing entity/relationship object, a newentity/relationship object can be created and the knowledge worker canbe provided the opportunity to establish an alias relationship betweenthe two entity/relationship objects. Some embodiments can include an“always alias” preference that automatically establishes an aliasrelationship between entities that have the same information fromdifferent electronic documents.

A sub-command of the quick-click command allows the knowledge worker tospecify how the entity object is inserted into the relationshiprepresentation space 305. For example, responsive to one sub-command,the entity object can be added to an evidence bundle object thatincludes a document page object refers to the electronic document pagethat sourced the selected string. This sub-command creates such anevidence bundle object if one is not already available in therelationship representation space 305. One embodiment defaults to thissub-command to allow the knowledge worker to quickly manipulate therelationship representation space 305 without undue manipulation of theGUI input devices.

Subsequent quick-click commands on text selected from the same documentpage object can cause entity objects to be added to the same evidencebundle object. The ‘add entity(s)’ procedure 1011 can also allow theknowledge worker to specify an initial position in the evidence panel501 to present the instance-representation of the newly inserted entityobject. Some embodiments automatically set the initial position of theinstance-representation. Some embodiments maintain an ordering aspectfor newly created entity/relationship objects (such as by positioningthe instance-representation of the newly added entity/relationshipobject in a non-overlapping position or by positioning theinstance-representation in a reading order (such as left-to-right ortop-to-bottom). Such embodiments reduce the knowledge worker's effortwhen manipulating the relationship representation space 305.

Once objects are added to the relationship representation space 305 thequick-click command process 1000 can post an event to refresh theevidence panel 501 and present instance-representations ofseparately-movable instance objects representing the newly added objectsin the relationship representation space 305. Once the command iscompleted, the quick-click command process 1000 completes through an‘end’ terminal 1013.

The operation of the ‘user-selected range as entity’ procedure 1009 andthe ‘add entity(s)’ procedure 1011 automatically determines an insertionposition and adds the selected text to the evidence panel 501 at thatposition (and automatically updates the representation spaces) as if theknowledge worker had copied the selected text from the text presentationwindow 900 window, activated the workspace window 500, selected theevidence panel 501 and pasted the copied text into the evidence panel501.

If the quick-click sub-command specifies the cursor-identifiedsub-command, a ‘cursor designated entity’ procedure 1015 is executedthat determines whether the knowledge worker's cursor positionintersects an identified string in the text presentation window 900, aword of text, or neither. If neither, the command can be ignored, or anerror message posted. Likewise, if the cursor position intersects a wordof text (as separated from surrounding text by white space orpunctuation, for example), the identified word can be automaticallyadded to the evidence panel 501 (and the representation spacesautomatically updated) by the ‘add entity(s)’ procedure 1011 aspreviously described.

If the quick-click sub-command specifies the entities-in-rangesub-command, an ‘entities in range’ procedure 1017 can be executed thatlocates all of the identified strings in the text presentation window900 that intersect with, or are completely included within, theknowledge worker's selected text (the selection defines the range).Using this sub-command, one or more identified strings can be passed tothe ‘add entity(s)’ procedure 1011 that automatically adds theidentified strings to the evidence panel 501 (and automatically updatesthe representation spaces) and then relates the inserted entity objects(or separately-movable instance objects representing a pre-existingentity objects) with an evidence bundle object. Thus, when the instancerepresentation space 303 is next presented (for example, in response toa evidence panel 501 update event), the instance-representation thatrepresents the newly added entity object will be presented within ainstance-representation that represents the evidence bundle object(indirectly through the instance representation space 303).

If the quick-click sub-command specifies theentities-and-relationships-in-range sub-command, an ‘entities in rangeand relationships’ procedure 1019 can be executed that locates all ofthe identified strings in the text presentation window 900 thatintersect with, or are completely included within, the knowledgeworker's selected text. In addition, the selected text can belinguistically processed to determine relationships between theidentified strings. With this sub-command one or more identified stringsand their relationships (as determined by the linguistic processing) canbe passed to the ‘add entity(s)’ procedure 1011. This sub-commandautomatically adds the identified strings to the evidence panel 501 (andautomatically updates the representation spaces), relates the relevantentity objects with an evidence bundle object as above, and furtherrelates the relevant entity objects with belief statement objectsresponsive to their linguistic relationships. Thus, when the instancerepresentation space 303 is next presented (for example, in response toa evidence panel 501 update event), the instance-representation of thenewly added entity objects (or separately-movable instance objectsrepresenting a pre-existing entity objects) will be presented asinstance-representations representing the belief statement objectswithin an instance-representation representing the newly added evidencebundle object.

The linguistic processing uses known techniques from computationallinguistics to process the sentences in, or surrounding, the selectedtext to determine relationships between the identified strings (forexample, but without limitation, such as “Person X has phone number Y”,or “Person X works at a company Y”, or “Person X is giving money toPerson Y”).

Note that the identified strings found by the electronic documentpreparation process 700 of FIG. 7 can be used by the cursor-identifiedsub-command, the entities-in-range sub-command, and theentities-and-relationships-in-range sub-command. This provides theknowledge worker with the ability to add the identified string(s) to therelationship representation space 305 with a single click. Without theidentified string the program would not be able to determine how manywords to copy nor which words to copy in response to a click. Note thatif the knowledge worker clicks on a non-highlighted word in the textpresentation window 900, quick-click will insert an entity object intothe relationship representation space 305 that represents that singlenon-highlighted word (or if that entity object is pre-existing, then aseparately-movable instance object representing the pre-existing entityobject will be added to the instance representation space 303.

The evidence bundle object instance 521 of FIG. 5 illustrates theevidence bundle object that would be produced by clicking on most of thehighlighted text strings in FIG. 9 in left-to-right top-to-bottom order.

Because the knowledge worker may sometimes wish to copy a phrase otherthan an identified string (or an identified string but with differentstarting or ending words than are included with the identified string),the knowledge worker can also select an arbitrary phrase (e.g., usingwell-known drag-select or any other method for selecting a sub-stringfrom a document) and then quick-click on the selected phrase to invokethe ‘user-selected range as entity’ procedure 1009. Knowledge workerselection of a phrase takes precedence over identified strings found by,for example, the electronic document preparation process 700. Thus, theselected phrase can be added to the relationship representation space305 as an entity object or a comment object, responsive to thesub-command of the invoked quick-click command.

All entity/relationship objects, evidence bundle objects and beliefstatement objects are editable (including those created usingquick-click operations) such that the knowledge worker can reorder thecontents of the object, add comment objects, combine entity/relationshipobjects into belief statement objects, and perform any other operationsthat the knowledge worker deems necessary in order to best represent thecomprehension state of the document collection.

Some embodiments optionally add new quick-click selected strings byautomatically recognizing the composite object that received previouslyselected strings from the same electronic document in which the newselected strings were found and selected.

FIG. 11 illustrates a user command dispatcher process 1100 that can beused to implement some of the graphical user interface commands for theworkspace window 500. The user command dispatcher process 1100 can beinvoked when the workspace window 500 is first presented, initiates at astart terminal 1101 and continues to a ‘detect command event’ procedure1103 that detects when the knowledge worker submits a command (such asby the press of a mouse button, a key, performance of a gesture, etc.).In many embodiments the command invocation can be detected by receivingan event. A ‘dispatch command’ procedure 1105 evaluates the detectedcommand and dispatches the command to a procedure that causes thecommand to be performed.

An ‘add document’ procedure 1107 adds an electronic document to thedocument collection and initiates any pre-processing that may be neededon that document (for example, by invoking the electronic documentpreparation process 700). In addition the ‘add document’ procedure 1107and/or the electronic document preparation process 700 can determine ifthe electronic document had been previously pre-processed by anout-of-date set of rules and, if so, can reprocess the electronicdocument with up-to-date rules.

A ‘create object’ procedure 1109 creates entity/relationship objects inthe relationship representation space 305, creates separately-movableinstance objects in the instance representation space 303 as needed, andupdates the belief representation space 307 responsive to the changedrelationship representation space 305. Examples of this class ofcommands include sub-commands or command modifiers of the cut-and-paste,drag-and-drop, insert comment, insert user-defined entity, andquick-click commands.

An ‘edit object’ procedure 1111 supports commands used by the knowledgeworker to edit properties of objects in one or more of therepresentation spaces (for example, to change a degree-of-interest valueor other property in the entity/relationship object, to change thedisplay coordinate property in a separately-movable instance object,etc.).

A ‘link objects’ procedure 1113 implements relationship commands thatallow the knowledge worker to change the relationships betweenentity/relationship objects. For example, this procedure could be usedto invoke a coupling command and/or an aliasing command functionality tocreate or destroy coupling relationships and/or alias relationships.

A ‘relate objects’ procedure 1114 establishes relationships betweenentity/relationship objects in the relationship representation space305, adjusts the separately-movable instance objects in the instancerepresentation space 303 that represent the objects as needed, andupdates the belief representation space 307 responsive to the changedrelationship representation space 305. Examples of this class ofcommands include sub-commands or command modifiers of the cut-and-paste,drag-and-drop, insert comment, insert user-defined entity, andquick-click commands. The ‘relate objects’ procedure 1114 can be invokedby the ‘create object’ procedure 1109 (after it creates a compositeobject) to form a relationship between the entity/relationship objectsrepresented by the manipulated instance-representations. In addition,the ‘relate objects’ procedure 1114 is invoked when the knowledge workeradds an entity/relationship object to an existing relationship (forexample, by adding an entity object to a composite object).

A ‘copy object’ procedure 1115 implements commands that allow theknowledge worker to add an instance-representation (that represents anentity/relationship object) by duplicating a separately-movable instanceobject in the instance representation space 303. Once the selectedcommand completes, the user command dispatcher process 1100 continuesback to the ‘detect command event’ procedure 1103 to await the nextcommand.

The user command dispatcher process 1100 can be used add, modify, alter,create, or destroy relationships resulting from composite objects.

While the user command dispatcher process 1100 was described in thecontext of an event driven and object-oriented graphical user interface,one skilled in the art would recognize that equivalent functionalitycould be provided using many other programming techniques.

Some of the commands handled by the user command dispatcher process 1100include associating entity/relationship objects within an evidencebundle object or a belief statement object; relating separately-movableinstance objects by a comment object, creating a coupling relationshipor alias relationship between entities; inserting, deleting ormodifying, an electronic document object, a document page object, acomment object, a composite object, an evidence bundle object, and abelief statement object within the relationship representation space305.

As has been previously discussed, entity/relationship objects can begrouped together within an evidence bundle object using quick-clickcommands or traditional cut-and-paste, and drag-and-drop commands. Onerelationship command can be invoked by the knowledge worker placing oneinstance-representation of a separately-movable instance object in closevertical proximity (for example, within a threshold distance) to aninstance-representation of a second separately-movable instance object.In some embodiments, the two entity/relationship objects represented bythe separately-movable instance objects can be combined into a newevidence bundle object (if neither of the entity/relationship objectswere already in an evidence bundle object). In a like manner the movedentity/relationship object can be added to an existing composite object.Further, composite objects can contain other composite objects such thatthe knowledge worker can combine belief statement objects and evidencebundle objects. The relationships of the objects in the relationshiprepresentation space 305 as manipulated by the knowledge worker can bethen used to generate the belief graph in the belief representationspace 307. One example of an implementation of a relationship command isillustrated by FIG. 12.

Some embodiments also detect when two instance-representations ofseparately-movable instance objects are placed in close horizontalproximity (for example, within the threshold distance) and can generatea belief statement object to assert a strong relationship between thetwo entity/relationship objects represented by the separately-movableinstance objects (such as by the belief statement object instance 510where, in this embodiment, the relationships/entities related by thebelief statement object are presented side-by-side, underlined, andwithin bookend delimiters). Entity/relationship objects that arecontained in the belief statement object have a stronger relationshipthan the relationship resulting when the entity/relationship objects arein an evidence bundle object and the strength of the relationship isreflected in the belief representation space 307.

Some entity/relationship objects have such a close relationship that theinstance-representations of both entity/relationship objects are alwayspresented together. In this situation the coupled entity/relationshipobjects can be placed in a coupling relationship. To define the couplingrelationship in one embodiment, the knowledge worker uses the cursor topoint to both instance-representations of the entity/relationship objectin turn and invokes a “coupling” command. For example, in FIG. 5, if theknowledge worker decides that whenever an instance-representation of thephone number “650-767-1265” is presented, that aninstance-representation of the name “Joe Jones” should also bepresented, the knowledge worker can point to bothinstance-representations in turn and invoke a coupling command to createa coupling relationship between the entity/relationship objectsrepresented by the instance-representations. An example of aninstance-representation that represents two entity/relationship objectsin a coupling relationship is the coupling relationship instance 517 inFIG. 5. The knowledge worker can, using the same selection process,instead invoke an aliasing command to create an alias relationshipbetween the selected entity/relationship objects.

Thus, the relationships between entity/relationship objects or otherobjects in the relationship representation space 305 can be representedby the spatial proximity and graphical presentation markers (such assurrounding boxes to indicate an evidence bundle object, or by addingbookend and an underline to indicate a belief statement object) in theevidence panel 501 of the corresponding instance-representations.

Additional objects can be added to a composite object (such as theevidence bundle object and the belief statement object) after thecomposite object has been created. Instance-representations representingthese additional objects can be positioned on or near theinstance-representation representing the composite object to specify howthe additional object is to be added to the composite object. Inaddition, the additional object may be added to, or used to create abelief statement object in an evidence bundle object if the additionalobject is also placed in close horizontal proximity to aninstance-representation representing an entity object or to aninstance-representation representing a belief statement object.

The knowledge worker can specify a degree-of-interest value for anyentity/relationship object. The degree-of-interest value indicates thedegree to which that relationship/entity is of interest to the knowledgeworker.

The workspace window enables commands for setting the degree-of-interestvalue of each entity/relationship object. Entity/relationship objectscan be initially given a degree-of-interest value that indicates “ofpossible interest”. The process used to present the belief graph in thesystem suggestions panel 503 varies the presentation of theinstance-representation based on the degree-of-interest value of theentity/relationship object to help the knowledge worker quickly identifyhigh interest entities. The presentation can be varied by the use ofcolor, size, shape, font, spatial relationship betweeninstance-representations of separately-movable instance objectsrepresenting the entity/relationship objects, etc. The knowledge workercan alter the degree-of-interest value by designating aninstance-representation of a separately-movable instance object and thenmodifying the contents of the entity/relationship object represented bythe separately-movable instance object.

FIG. 12 illustrates a relationship command process 1200 that can beinvoked as part of a GUI command detection process and that initiates ata start terminal 1201. A ‘detect select-drag operation’ procedure 1203detects whether the pointing device has selected and is currentlydragging an instance-representation. If not, the relationship commandprocess 1200 simply returns (not shown). If so, the relationship commandprocess 1200 continues to a ‘detect drop’ procedure 1205 that determineswhen the knowledge worker drops the dragged instance-representation.When the instance-representation is dropped, the relationship commandprocess 1200 continues to a ‘compute distance to nearest instance’procedure 1207 that calculates a distance vector from the point of dropto other instance-representations and selects the nearest of the otherinstance-representations. Once the distance vector is calculated, a‘compute distance to nearest instance’ procedure 1207 then can determinewhether the nearest instance-representation is within a thresholddistance.

If the length of the shortest vector is outside a threshold distance atthe time of the drop, the relationship command process 1200 continues toa ‘modify presentation position of dropped separately-movable instanceobject’ procedure 1211 that changes the presentation position of theseparately-movable instance object represented by the draggedinstance-representation. The relationship command process 1200 thenexits through an end terminal 1213.

If the length of the shortest vector is within the threshold distance atthe time of the drop, the relationship command process 1200 continues toa ‘determine target boundary’ procedure 1215 that can determine whichboundary of the nearest instance-representation is nearest to the droppoint. A ‘create/modify composite object’ procedure 1217 then,responsive to which boundary of the nearest instance-representation isnearest the drop point, can create/modify a composite object—thus, thetwo instance-representations appear to “snap” together. If the nearestinstance-representation is an entity object, a new composite object canbe created that includes the entity object and the entity/relationshipobject represented by the dragged instance-representation. If thenearest instance-representation is a composite object theentity/relationship object represented by the draggedinstance-representation can be added to the composite object or to anentity/relationship object bundled with the composite object. Thestrength of relationship created by the ‘create/modify composite object’procedure 1217 (that is, whether the relationship is represented by anevidence bundle object, or a belief statement object) can be responsiveto which border is nearest to the drop point. In some embodiments thedrop point is the cursor location in the evidence panel 501 at the timethe instance-representation is dropped.

In one embodiment, the positioning is such that operations related tostatement relationships are given priority over evidence relationships.In this embodiment horizontal alignments are given priority oververtical alignments and thus belief statement object operations arepreferred over evidence bundle object operations (that is invoked by avertical alignment).

In some embodiments, when the instance-representation is in the processof being dragged, presentation aspects of non-draggedinstance-representations can change when the position of the draggedinstance-representation is sufficiently close to the non-draggedinstance-representation. One embodiment changes the presentation aspectof the non-dragged instance-representations by highlighting thoseinstance-representations (in an identifiable manner) to distinguish nearinstance-representations from instance-representations that are notsufficiently near to the dragged instance-representation. In addition,with respect to highlighted near instance-representations,instance-representations within the near instance-representation (thatrepresent entity/relationship objects related by the highlighted nearinstance-representation) can also be highlighted responsive to whetherthey can be operated on by the dragged instance-representation. Thus,when the knowledge worker desires to add to an existing evidence bundleobject, he/she can drop a dragged instance-representation anywhere inthe existing order of entity/relationship objects within the evidencebundle object and can drop the instance-representation before, after, orin between (in the vertical dimension) the instance-representationsrepresenting the entity/relationship objects already in the evidencerelationship. In a similar manner the knowledge worker can place thedragged instance-representation anywhere in the ordering of a beliefstatement object by dropping the dragged instance-representation before,after, or in between (in the horizontal dimension) theinstance-representations representing the entity/relationship objectsalready in the statement relationship.

The threshold distance can be a multidimensional vector where theselection of the strength of the relationship can be responsive toweighted values of one or more of the vector's elements.

The previous description is directed to one embodiment for selecting asub-command as applied to one instance-representation that is responsiveto the relative position of a dropped instance-representation to theone. Thus, the determination of which border is nearest to the droppoint is one embodiment of specifying spatial relationships (such asangular, or distance relationships in two dimensional display space, orof relationships in a velocity/position space) that can be used todistinguish one spatial relationship from another. Once the spatialrelationship between the two instance-representations is determined, acommand, sub-command, and/or command modifier can be invoked to performan operation (responsive to the determined spatial relationship) on theentity/relationship objects in the relationship representation space 305(that are represented by the two instance-representations). Theoperation can create a new composite object or alter an existingcomposite object.

Some embodiments are configured such that if the drop point is near aleft or right edge of a target instance-representation the droppedentity/relationship object can be added to, or creates, a beliefstatement object within the entity/relationship object represented bythe target instance-representation. In this embodiment, if the droppoint is near a top or bottom edge of the targetinstance-representation, the dropped entity/relationship object can beadded to, or creates, an evidence bundle object. If the drop point isnear two edges, some embodiments have a preference as to which (left,right, top, bottom) edge, or pair of edges are preferred (such that thecorresponding operation has a higher priority over the operationsassociated with the other edges). Some embodiments default to selecting“leaf” structures in the relationship representation space 305 fromwhich to measure the vector. In other words, after a horizontal snap,the technology ensures that the entity/relationship object representedby the dragged instance-representation and the chosen stationary leafentity/relationship object are in a statement relationship byassociating these entity/relationship objects within the same beliefstatement object (and by creating a new belief statement object ifneeded). Likewise, after a vertical snap, the technology ensures thatthe entity/relationship object represented by the draggedinstance-representation and the chosen stationary leafentity/relationship object are in a evidence relationship by associatingthese entity/relationship objects within the same evidence bundle object(and by creating a new evidence bundle object if needed).

Other embodiments use various techniques well known to one skilled inthe art to post a command selection responsive to a near drop eventwhere one of the possible command selections is preferred over anotherof the possible command selections

One skilled in the art, after reading the previously disclosedtechnology will understand that the document collection can be compactlysummarized for/by the knowledge worker from information in therelationship representation space 305 and the belief representationspace 307. One example of such a summarization is the construction of atimeline story related to the entities of high interest. For example,the user may position the evidence bundles in a left-to-rightarrangement, sorted by date, in order to view a sequence of events inchronological order.

As used herein, a procedure is a self-consistent sequence of steps thatcan be performed by logic implemented by a programmed computer,specialized electronics or other circuitry or a combination thereof thatlead to a desired result. These steps can be defined by one or morecomputer instructions. These steps can be performed by a computerexecuting the instructions that define the steps. Further, these stepscan be performed by circuitry designed to perform the steps. Thus, theterm “procedure” can refer (for example, but without limitation) to asequence of instructions, a sequence of instructions organized within aprogrammed-procedure or programmed-function, a sequence of instructionsorganized within programmed-processes executing in one or morecomputers, or a sequence of steps performed by electronic or othercircuitry, or any logic or combination. In particular one skilled in theart after reading this specification would understand how to implement,without undue experimentation, a relationship space edit logic, apresentation logic, a belief space access logic, a belief space editlogic, a degree-of-interest logic, a first update logic, a scoringlogic, a rule logic, a quick-click command invocation logic, a userinterface logic, a comparison logic, a linguistic processing logic, acommand detection logic, a selection logic, and an instancerepresentation space edit logic.

One skilled in the art will understand that the network transmitsinformation (such as informational data as well as data that defines acomputer program). The information can also be embodied within acarrier-wave. The term “carrier-wave” includes electromagnetic signals,visible or invisible light pulses, signals on a data bus, or signalstransmitted over any wire, wireless, or optical fiber technology thatallows information to be transmitted over a network. Programs and dataare commonly read from both tangible physical media (such as a compact,floppy, or magnetic disk) and from a network. Thus, the network, like atangible physical media, is a computer-usable data carrier.

One skilled in the art will understand that the technology improves theability of a knowledge worker to discover, remember, and summarize thecomprehension state of a document collection.

From the foregoing, it will be appreciated that the technology has(without limitation) the following advantages:

-   -   1. Quick-click entity extraction reduces the time necessary to        identify and select entities;    -   2. Automatic linguistically-supported creation of relationships        reduces the time necessary to identify and specify        relationships;    -   3. Snap-together relationship commands reduce the time necessary        to specify a relationship;    -   4. Interactive editing of the comprehension state of a document        collection through a user interface based on spatial grouping of        entities and relationships reduces the time necessary to analyze        and record information from a document collection;    -   5. Automated recommendation of relationships and documents for        further investigation from analysis of the belief graph helps        guide the knowledge worker to relevant areas of analysis;    -   6. Ability to couple entities such that the entities are always        presented together assists the knowledge worker with making        inferences and thus reduces the probability that a relationship        will be overlooked;    -   7. Discovery of linked entities using distinctive highlighting        of shared relationships reduces the knowledge worker's effort        when examining presented entities and relationships;    -   8. Entity highlighting and dimming reduces the knowledge        worker's effort when examining presented entities and        relationships and when making inferences based on the presented        information.    -   9. Highlighting of entities in a document based on        degree-of-interest values specified by the knowledge worker        simplifies the knowledge worker's task when reading a document;    -   10. Assists the knowledge worker in remembering/locating the        source of entity information and entity details, and remembering        relationships; and    -   11. Promotes sharing the comprehension state of the document        collection between knowledge workers by providing an explicit        representation of the comprehension state that can be shown to        or given to other knowledge workers in whole or in part.

The claims, as originally presented and as they may be amended,encompass variations, alternatives, modifications, improvements,equivalents, and substantial equivalents of the embodiments andteachings disclosed herein, including those that are presentlyunforeseen or unappreciated, and that, for example, may arise fromapplicants/patentees and others.

It will be appreciated that various of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be desirablycombined into many other different systems or applications. Also thatvarious presently unforeseen or unanticipated alternatives,modifications, variations or improvements therein may be subsequentlymade by those skilled in the art which are also intended to beencompassed by the following claims. Unless specifically recited in aclaim, steps or components of claims should not be implied or importedfrom the specification or any other claims as to any particular order,number, position, size, shape, angle, color, or material.

1. A computer controlled method comprising: extracting separately-movable instance objects representing entity/relationship objects from a collection of documents; displaying, in a workspace window, a story-organized visual presentation that represents a comprehension state by organizing a plurality of instance representations into entity-clickable sentences for easy visual comprehension, the story-organized visual presentation including a plurality of instance representations of the separately-movable instance objects representing entity/relationship objects in said story-organized visual presentation, the instance representations being displayed such that instance representations having relationships to each other are abuttedly coupled to each other; altering the story-organized visual presentation responsive to customized manipulation from a user of the story-organized visual presentation of the comprehension state comprising changes to the position and abutment relationships of a first instance-representation relative to other instance representations; accessing a belief graph, said belief graph comprising a plurality of fact nodes that correspond to the plurality of instance representations and a plurality of weighted edges that correspond to the relationships between each fact node of the plurality of fact nodes, one of said plurality of weighted edges connecting a first of said plurality of fact nodes and a second of said plurality of fact nodes, each instance representation of the plurality of instance representations being associated with a fact node of the plurality of fact nodes; altering said belief graph responsive to the user customized altering of said story-organized visual presentation; presenting in the workspace window, the entity-clickable sentences, which include text strings with separately-movable subparts, the separately movable subparts including the plurality of instance representations; identifying the text strings represented in the entity-clickable sentences in the workspace window based on a level of interest corresponding to relationships between the instance representations in the belief graph; and modifying, responsive to the altering of the belief graph, a weighted edge of the plurality of weighted edges, wherein the presenting is also responsive to said belief graph as altered, the entity-clickable sentences visibly mark the plurality of instance representations and allow the knowledge worker to click on an instance representation in the sentences so the knowledge worker can see where else the instance representation is used in the story-organized visual presentation, the story-organized visual presentation orders story information based on the preferences of the knowledge worker and allows the knowledge worker to determine the order of instance representations displayed in sentence form, and the text strings have visual indicators that are based on at least one of a level of interest to the knowledge worker and an entity type.
 2. The computer controlled method of claim 1, further comprising: assigning a degree-of-interest value to said first entity/relationship object; generating a snap-together sentence by abutting two or more instance representations and/or comments and updating said belief graph and adding comments to the snap-together sentence to provide a logical and cohesive representation of the instance representation of the customized belief graph; automatically creating a first weighted edge based on the snap-together sentence in the belief graph between the entity/relationship objects and the corresponding instance representations; automatically creating a second weighted edge that is less weighted than the first weighted edge based on the snap-together paragraph in the belief graph between the entity/relationship objects and the corresponding instance representation; computing scores to entity/relationship objects that have not been given scores by the user based on the proximity in the belief graph between each unscored entity/relationship object and any nearby scored entity/relationship object; and updating the story-organized visual presentation to communicate the final scores in colors based on its corresponding final score to the user.
 3. The computer controlled method of claim 2, further comprising: recognizing one or more identified strings within an ordered set of text strings; and presenting a presentation set of said ordered set of text strings where one of said one or more identified strings within said presentation set is distinguished from a second subset of said presentation set responsive to said score.
 4. The computer controlled method of claim 1, further comprising: inserting a second entity/relationship object into said story-organized visual presentation, said second entity/relationship object selected from one or more of the group consisting of an electronic document object, a comment object, a document page object, a composite object, an evidence bundle object, an entity object and a belief statement object.
 5. The computer controlled method of claim 1, wherein a first relationship between said first entity/relationship object and a second entity/relationship object in said story-organized visual presentation is represented by spatial proximity in said workspace window of said first instance-representation of said first separately-movable instance object representing said first entity/relationship object with a first instance-representation of a second separately-movable instance object representing said second entity/relationship object.
 6. The computer controlled method of claim 5, wherein said first relationship is selected from one or more of a group consisting of a coupling relationship, a statement relationship, an alias relationship, and an evidence relationship.
 7. The computer controlled method of claim 1, further comprising: applying at least one rule to an ordered set of text strings to recognize one or more identified strings within said ordered set of text strings.
 8. The computer controlled method of claim 1, further comprising: assigning an entity-categorization to said first entity/relationship object.
 9. The computer controlled method of claim further comprising: recognizing one or more identified strings within an ordered set of text strings; accessing an electronic document containing at least one of said ordered set of text strings; and presenting a portion of said electronic document with said at least one of said one or more identified strings distinguished.
 10. The computer controlled method of claim 1, further comprising modifying said story-organized visual presentation by adding a second entity/relationship object responsive to a selected string.
 11. The computer controlled method of claim 10, wherein modifying is responsive to a quick-click command invocation and said selected string is one or more identified strings from a presentation set of an ordered set of text strings.
 12. The computer controlled method of claim 1, further comprising establishing a relationship between said first entity/relationship object with a second entity/relationship object.
 13. The computer controlled method of claim 1, further comprising assigning a degree of-interest value to said first entity/relationship object.
 14. The computer controlled method of claim 1, further comprising: selecting a first instance-representation using a pointing device; identifying a fact node of the plurality of fact nodes that is associated with the first instance-representation in the relationship data structure; modifying a color, text, icon or other visual characteristics of all of the plurality of instance-representations in the text strings in the workspace window that are associated with fact nodes that are connected by a weighted edge to the identified fact node when a strength of relationship is larger than a predetermined value.
 15. An apparatus having a central processing unit (CPU) and a memory coupled to said CPU comprising: an instance object extracting logic that extracts separately-movable instance objects representing entity/relationship objects from a collection of documents; a relationship space edit logic configured to display, in a workspace window, a story-organized visual presentation that represents a comprehension state by organizing a plurality of instance representations into entity-clickable sentences for easy visual comprehension, the story-organized visual presentation including a plurality of instance representations of the separately-movable instance objects representing entity/relationship objects in said story-organized visual presentation, the instance representations being displayed such that instance representations having a predetermined relationship to each other are abuttedly coupled to each other, and alter the story-organized visual presentation responsive to customized manipulation from a user of the story presentation of the comprehension state comprising changes to the position and abutment relationships of a first instance-representation relative to other instance representations; a belief space access logic configured to access a belief graph, said belief graph comprising a plurality of fact nodes that correspond to the instance representations and a plurality of weighted edges that correspond to the relationships between each fact node of the plurality of fact nodes, one of said plurality of weighted edges connecting a first fact node of said plurality of fact nodes and a second fact node of said plurality of fact nodes, each instance representation of the plurality of instance representations being associated with a fact node of the plurality of fact nodes; a belief space edit logic configured to alter said belief graph responsive to the user customized altering of the belief space access logic and said story-organized visual presentation as altered by the relationship space edit logic; a text presentation window logic, in the workspace window, configured to present text so that text strings represented in the entity-clickable sentences are identified based on a level of interest corresponding to relationships between the instance representations in the belief graph, the text strings having separately-movable subparts, the separately movable subparts including the plurality of instance representations; and a belief graph modifying logic that, responsive to the altering of the believe graph, modifies a weighted edge of the plurality of weighted edges, wherein the presentation logic is also responsive to the belief space edit logic and said belief graph, the entity-clickable sentences visibly mark the plurality of instance representations and allow the knowledge worker to click on an instance representation in the sentences so the knowledge worker can see where else the instance representation is used in the story-organized visual presentation, the story-organized visual presentation orders story information based on the preferences of the knowledge worker and allows the knowledge worker to determine the order of instance representations displayed in sentence form, and the text strings have visual indicators that are based on at least one of a level of interest to the knowledge worker and an entity type.
 16. The apparatus of claim 15, further comprising: a degree-of-interest logic configured to assign a degree-of-interest value to said first entity/relationship object responsive to the relationship space edit logic; a snap-together sentence generating logic that generates a snap-together sentence by abutting two or more instance representations and/or comments and updates said belief graph and adding comments to the snap-together sentence to provide a logical and cohesive representation of the instance representations of the customized belief graph; a first weighted edge logic that automatically creates a first weighted edge based on the snap-together sentence in the belief graph between the entity/relationship objects and the corresponding instance representations; a second weighted edge logic that automatically creates a second weighted edge that is less weighted than the first weighted edge based on the snap-together paragraph in the belief graph between the entity/relationship objects and the corresponding instance representation; a computing score logic that computes scores to entity/relationship objects that have not been given scores by the user based on the proximity in the belief graph between each unscored entity/relationship object and any nearby scored entity/relationship object; and an updating logic that updates the story-organized visual presentation to communicate the final scores in colors based on its corresponding final score to the user.
 17. The apparatus of claim 15, wherein a first relationship between said first entity/relationship object and a second entity/relationship object in said story-organized visual presentation is represented by spatial proximity in said workspace window of said first instance-representation of said first separately-movable instance object representing said first entity/relationship object with a first instance-representation of a second separately-movable instance object representing said second entity/relationship object.
 18. The apparatus of claim 17, wherein said relationship is selected from one or more of a group consisting of a coupling relationship, a statement relationship, an alias relationship, and an evidence relationship.
 19. The apparatus of claim 15, further comprising: the relationship space edit logic further configured to insert a second entity/relationship object into said story-organized visual presentation, said second entity/relationship object selected from one or more of the group consisting of an electronic document object, a comment object, a document page object, a composite object, an evidence bundle object, an entity object and a belief statement object.
 20. The apparatus of claim 15, further comprising: a rule logic configured to apply at least one rule to an ordered set of text strings to recognize one or more identified strings within said ordered set of text strings.
 21. The apparatus of claim 15, wherein the relationship space edit logic is further configured to assign an entity-categorization to said first entity/relationship object.
 22. The apparatus of claim 15, further comprising: a rule logic configured to recognize one or more identified strings within an ordered set of text strings; and wherein the presentation logic is further configured to present a presentation set of said ordered set of text strings where one of said one or more identified strings within said presentation set is distinguished from a second subset of said presentation set.
 23. The apparatus of claim 15, wherein the relationship space edit logic is further configured to modify said story-organized visual presentation by adding a second entity/relationship object responsive to a selected string.
 24. The apparatus of claim 15, wherein the relationship space edit logic is further configured to establish a relationship between said first entity/relationship object with a second entity/relationship object.
 25. A computer program storage medium on which is stored a program that causes a computer to perform a method comprising: extracting separately-movable instance objects representing entity/relationship objects from a collection of documents; displaying, in a workspace window, a story-organized visual presentation that represents a comprehension state by organizing a plurality of instance representations into entity-clickable sentences for easy visual comprehension, the story-organized visual presentation including a plurality of instance representations of the separately-movable instance objects representing entity/relationship objects in said story-organized visual presentation, the instance representations being displayed such that instance representations having a predetermined relationship to each other are abuttedly coupled to each other; altering the story-organized visual presentation responsive to customized manipulation from a user of the story-organized visual presentation of the comprehension state comprising changes to the position and abutment relationships of a first instance-representation relative to other instance representations; accessing a belief graph, said belief graph comprising a plurality of fact nodes that correspond to the instance representations and a plurality of weighted edges that correspond to the relationships between each fact node of the plurality of fact nodes, one of said plurality of weighted edges connecting a first of said plurality of fact nodes and a second of said plurality of fact nodes, each instance representation of the plurality of instance representations being associated with a fact node of the plurality of fact nodes; altering said belief graph responsive to the user customized altering of said story-organized visual presentation; presenting in the workspace window, the entity-clickable sentences, which include text strings with separately-movable subparts, the separately movable subparts including the plurality of instance representations; identifying text strings represented in the entity-clickable sentences in the workspace window based on a level of interest corresponding to relationships between the instance representations in the belief graph; and modifying, responsive to the altering of the belief graph, a weighted edge of the plurality of weighted edges, wherein the presenting is also responsive to said belief graph as altered, the entity-clickable sentences visibly mark the plurality of instance representations and allow the knowledge worker to click on an instance representation in the sentences so the knowledge worker can see where else the instance representation is used in the story-organized visual presentation, the story-organized visual presentation orders story information based on the preferences of the knowledge worker and allows the knowledge worker to determine the order of instance representations displayed in sentence form, and the text strings have visual indicators that are based on at least one of a level of interest to the knowledge worker and an entity type.
 26. The computer program storage medium of claim 25, further comprising: assigning a degree-of-interest value to said first entity/relationship object; generating a snap-together sentence by abutting two or more instance representations and/or comments and updating said belief graph and adding comments to the snap-together sentence to provide a logical and cohesive representation of the instance representations of the customized belief graph; automatically creating a first weighted edge based on the snap-together sentence in the belief graph between the entity/relationship objects and the corresponding instance representations; automatically creating a second weighted edge that is less weighted than the first weighted edge based on the snap-together paragraph in the belief graph between the entity/relationship objects and the corresponding instance representation; computing scores to entity/relationship objects that have not been given scores by the user based on the proximity in the belief graph between each unscored entity/relationship object and any nearby scored entity/relationship object; and updating the story-organized visual presentation to communicate the final scores in colors based on its corresponding final score to the user.
 27. The computer program storage medium of claim 25, further comprising: inserting a second entity/relationship object into said story-organized visual presentation, said second entity/relationship object selected from one or more of the group consisting of an electronic document object, a comment object, a document page object, a composite object, an evidence bundle object, an entity object and a belief statement object.
 28. The computer program storage medium of claim 25, further comprising: applying at least one rule to an ordered set of text strings to recognize one or more identified strings within said ordered set of text strings.
 29. The computer program storage medium of claim 25, further comprising: recognizing one or more identified strings within an ordered set of text strings; accessing an electronic document containing at least one of said ordered set of text strings; and presenting a portion of said electronic document with said at least one of said one or more identified strings distinguished.
 30. The computer program storage medium of claim 25, further comprising modifying said story-organized visual presentation by adding a second entity/relationship object responsive to a selected string.
 31. The computer program storage medium of claim 25, further comprising establishing a relationship between said first entity/relationship object with a second entity/relationship object. 